Efficient measurements, purification, and bounds on the mutual information 
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When a measurement is made on a quantum system in which classical information is encoded, 
the measurement reduces the observers average Shannon entropy for the encoding ensemble. This 
reduction, being the mutual information, is always non-negative. For efficient measurements the 
state is also purified; that is, on average, the observers von Neumann entropy for the state of the 
system is also reduced by a non-negative amount. Here we point out that by re-writing a bound 
derived by Hall [Phys. Rev. A 55, fOO (1997)], which is dual to the Holevo bound, one finds that for 
efficient measurements, the mutual information is bounded by the reduction in the von Neumann 
entropy. We also show that this result, which provides a physical interpretation for Hall's bound, 
may be derived directly from the Schumacher- Westmoreland- Wootters theorem [Phys. Rev. Lett. 
76, 3452 (1996)]. We discuss these bounds, and their relationship to another bound, valid for 
efficient measurements on pure state ensembles, which involves the subentropy. 
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In what follows we will be concerned with the situation 
in which one observer, the sender, transmits information 
to another observer, the receiver, by encoding that in- 
formation in a quantum system and having the receiver 
make a subsequent measurement on the system. It is 
useful at this point to define all our terminology and no- 
tation regarding this information transmission process. 

To encode the information in the quantum system the 
sender uses an alphabet consisting of a set of possible 
states, and prepares the system in one of these states, 
Pi, with probability Pi. The set of states, along with 
their respective probabilities is referred to as the encod- 
ing, or the ensemble, and we will denote it by e = {Pi, pi}. 
When the system has been prepared by the sender, the 
state-of-knowledge of the receiver regarding the system 
i s P = Tlii Pi Pi- We will always denote the dimension of 
the system used for encoding by N, and we will refer to 
p as the ensemble state. 

The measurement made by the receiver is described by 
a set of operators, Aj, such that Ylj A^Aj = I 0,0,01. 
We will denote the measurement by M. = {Aj}, and 
where convenient denote the operators AjAj as Ej. For 
efficient measurements, with which we will be concerned 
in the following unless otherwise stated, each of the oper- 
ators Aj corresponds to a measurement outcome, and the 
outcomes are therefore labeled by j. The final state of 
the system from the point of view of the observer, hav- 
ing obtained the outcome j, is p'j = AjpA^/Qj, where 
Qj = Tr[Ejp] is the probability that outcome j will re- 
sult. For clarity we will denote probability densities over 
i as P, and those over j as Q. 

The amount of information transmitted to the receiver 
in the process of preparation and measurement, which we 
will refer to as Aij, is given by the mutual information, 
H(I:J), between the preparation, indexed by i, and the 
outcomes, indexed by j Thus 

AJ; = H(I:J) = H[P t ] - QjH[Pmi (1) 



where H is the Shannon entropy, and P(i\j) is the re- 
ceiver's probability density for the prepared state, after 
having received outcome j. (That is, the receiver's final 
state-of-knowledge about which state was initially pre- 
pared.) The mutual information is thus the average dif- 
ference between the receiver's initial information about 
the preparation, and her final information after the mea- 
surement. We denote this by A/j to reflect this fact, 
with the subscript indicating that it constitutes informa- 
tion about the initial preparation. The maximum of A/j 
over all measurements, for a fixed encoding, is referred 
to as the accessible information of the encoding , and 
we will denote this by A/ acc . 

The celebrated Holevo bound provides a limit to the 
accessible information of an encoding 

H II The 

Holevo bound is 



(2) 



where S[p] denotes the von Neumann entropy of p. 

One can also a problem which may be viewed as being 
complementary to that of finding the accessible informa- 
tion; that of obtaining the maximum of A/; given that 
it is the receiver's measurement and the ensemble state 
p which are fixed, and it is instead the sender which has 
the ability to use any encoding consistent with p. Hall 
has shown that it is possible to use Holevo's bound, along 
with a duality relation between encodings and measure- 
ments, which he refers to as source duality, to derive a 
bound on Aij for this case. Hall's dual Holevo bound 



AliKSlpj-^QjS 



\/P e jVp 

Qi 



(3) 



The Holevo bound and (as we will show) the dual Holevo 
bound, may both be derived directly from the more gen- 
eral bound obtained by Schumacher, Westmoreland and 
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Wootters in 1996 We state this theorem now, and 
will return to it later. 

Theorem [Schumacher- Westmoreland- Wootters] : The 
information transmitted from sender to receiver, A/, 
when the sender uses the encoding e, and the receiver 
uses measurement M, is bounded such that 



A/i < S[p}-J2 p iSlPi} 
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(4) 



where all quantities are as defined above, and a new quan- 
tity is introduced, being p'^ , which is the final state that 
the receiver would have had, if she knew that the initial 
state was pi. Thus p'j i — Aj piJV- / Q{j\i) , where Q(j\i) 
is naturally the probability density for the measurement 
outcomes, given that the initial state is pi. Because of 
the final term on the right hand side of this inequality, 
to which we will return later, this bound is, in general, 
stronger than the Holevo bound. 

While A/j quantifies the information which the ob- 
server obtains about the initial preparation, there ex- 
ists another quantity which can be said to characterize 
the average amount of information which the receiver ob- 
tains about the final state which she is left with after the 
measurement [13 E3 We will denote this by AI f , the 
expression for which is 



(5) 



This is the average difference between the receiver's ini- 
tial von Neumann entropy of the quantum system, and 
her final von Neumann entropy. This quantity is useful 
when considering quantum state preparation and, more 
generally, quantum feedback control [lOj. 

While we have introduced A/i and A/f in terms of 
initial states and final states, the former is not really 
any more connected with initial states than it is with 
final states, since the Shannon entropy of the ensemble 
after measurement is independent of whether it is written 
in terms of the initial states or the final ones. A more 
fundamental difference between A/ and A/f is that the 
former is the average change in the observers Shannon 
entropy regarding the ensemble, where as the latter is the 
average change in the observers von Neumann entropy 
regarding the overall state of the quantum system. That 
is, A/ = (AH{e)) and A/ f = (AS(p(e))). 

We now ask, is there a relationship between the two 
kinds of information gathering, A/ and A/f? It turns 
out that the answer to this question is yes: the former 
is bounded by the latter (that is, (AH) < (AS)), and 
this is readily shown by observing that it is actualy an 
alternative form for Hall's dual to Holevo's bound. To 
do this one notes that if we use the polar decomposition 



and define Bj - 
which appear in the dual bound are 

while the final states are 



' 'Ejp, then the Hermitian operators 
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But B)Bj and BjB] have the same eigenvalues [l6|. 
Thus since the von Neumann entropy is only a function 
of the eigenvalues, we can replace ^fpE^^fp with Qjp'j in 
the original expression for the dual bound, and the result 



A/ < A/f. 



(8) 



One can interpret this as saying that an observer cannot 
learn more about the classical information encoded in a 
quantum system than she learns about the state of the 
quantum system. This provides a physical interpretation 
for Hall's dual bound. It is also worth noting, as was 
pointed out by Hall J9J , that this bound is only saturated 
when all the operators Ej commute. 

The above result may also be obtained from the SWW 
theorem. To do this one first re-writes the second and 
fourth terms of the RHS of Eq.Q, using the fact that 
QjPm = PiQ(j\i): 

i j i 



(9) 



theorem 15 1 to write A; 



where Uj is unitary, 



where Ala is the information that would have been ob- 
tained about the final state if the initial state had been 
Pi. This gives 

A/ < SW-^PiMH-^QiSty]. (10) 



Now, since Nielsen has shown that A/f is always non- 
negative (see also H3), the RHS is maximized when 
the A/k are zero for all i. Since this is true for all pure 
state ensembles, the result is the bound given in Eq.lJSJ. 

One consequence of Eq.JSJ) is that, if we choose an 
ensemble which has the maximal accessible information 
for a fixed p, we can only obtain all this information if 
all the final states are pure. As SWW point out in their 
paper, measurements which leave the final state impure, 
leave some information in the system. That is, if the 
final state is mixed, in general it depends on the initial 
ensemble, and as a result subsequent measurements can 
obtain further information about the initial preparation, 
whereas this is not possible if the final state is pure. 

For a given p not all ensembles have an accessible in- 
formation equal to S[p\. We may ask then, if it is possible 
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for measurements which leave the final state impure to 
extract all the accessible information from these encod- 
ings. In fact, this is only possible if the encoding satisfies 
special conditions; in general, incomplete measurements 
will not even extract the accessible information from an 
ensemble. To see this, consider the final states, p'j, which 
result from the measurement. Each of these consists of 
an ensemble, Sj over the states p^u, introduced above. In 
particular, 

i 

Since these ensembles consist of states indexed by i, they 
can, in general, be measured to obtain further informa- 
tion about the initial preparation. Since the accessible 
information is the maximal amount of information that 
can be obtained about i by making measurements, we 
have the inequality 

AJi(e,M) < A/ acc (e) - £ Q j A/ acc (e,). (12) 

j 

Thus, AIi(e,A4) can only be equal to A/ acc (£) if the 
AJ acc (ej) are zero for all j. If p'j is pure, then A/ acc (£j) 
is zero. If p'j is not pure, then the accessible information 
of £j is only zero if, for any given j, the p^u are the same 
for all i. A little algebra shows that this is only true if 

Pa^PiPaj - <>,',, I' \ /'/•/' \ =0, V i,k , (13) 

for some non-negative real numbers ajjy , where Pa is 
a projector onto the support of the operator Aj. This 
means that for a measurement to extract all the accessi- 
ble information, all the coding states pj must be identical, 
up to a multiplier, on the supports of the operators Aj, 
separately for every j. 

For pure-state ensembles it is easy to see the effect of 
the conditions given by Ea. (|13fl . Consider merely the j 
for which the corresponding Aj has the support with the 
largest dimension, and call this dimension M max . Then 
the effect of Eq. (|13fl for this j alone is simply to limit the 
dimension of the space from which the pure states in the 
ensemble can be drawn to N — M max + 1 . The accessible 
information of pure-state ensembles which satisfy Eq. I|13|) 
is therefore bounded by ln(N — M max + 1). 

As was noted by SWW, the expression in the square 
brackets in Eq.Q is the Holevo x quantity for the ensem- 
ble £j which results from measurement outcome j. Thus 
their bound may be written as 

A/i^H-^fey. (w) 

j 

Now, x[ £ j] is the Holevo bound on the information that 
the receiver could extract when making a subsequent 
measurement after obtaining result j. The SWW bound 
is therefore very interesting because it shows that, if the 
initial ensemble e is chosen so that its accessible infor- 
mation is maximal (ie. equal to x( £ ))i then the infor- 
mation obtained by an incomplete measurement will be 



reduced by the maximal amount of information which 
could be accessible from the final ensembles Ej, and not 
merely the actual information available in these ensem- 
bles, which would imply the bound given in Ea. <|12[) . In 
general, therefore, there is a gap between the information 
lacking in an incomplete measurement, and that which 
can be recovered by subsequent measurements. It is nat- 
ural to ask therefore if this is true for all ensembles. That 
is, whether the inequality in Eq. (|12fl can be strengthened 
by replacing the final term on the RHS by the final term 
in the RHS of Eci. (|14fl . However, this is not the case. As 
the following theorem shows, for pure-state ensembles a 
tight upper bound is obtained by replacing the final term 
in Eq. (|12H by the average of the subentropies of the final 
states 17], rather than the corresponding Holevo quan- 
tities. 

Theorem: For an initial pure-state ensemble e, and a 
general measurement M, one has 

AJiKAI^-^QjQlffj], (15) 

3 

where Q[] is the subentropy as defined by Jozsa, Robb 
and Wootters (JRW) [13, and this bound is tight in the 
sense that there exists a pure-state ensemble which sat- 
urates the inequality. 

Proof. If the initial ensemble e is pure, then the final 
ensembles £j are also pure. As a result, the accessible 
information of each of these ensembles is bounded below 
by Q[ptj ] [rT|- We can therefore replace the final sum in 
Eq.ljnf by £V QjQ[p'j\, which gives Eq.JTSJ. 

That the bound can be achieved can be shown by cal- 
culating A/j for the uniform ensemble over pure states, 
being the unique distribution over pure states which is 
invariant under unitary transformations. In this case the 
ensemble state is given by 

P = f\1>)Md\1>) = ±, (16) 

where d\ip) represents inte grat ion over the unitarily in- 
variant, or Haar, measure Il8l. The accessible informa- 
tion is given by Q[I/N] [l2|. The information obtained 
by a general measurement may be calculated directly: 

AJi = H[Qj]-J H[Q( 3 M)]d\i>) 

= lnTV + ^Tr^-] / <V^#) HWj WW) 

3 

= Qii/m-J^QiQW (i7) 

j 

where the integral in the second line is performed using 
the techniques in Ref. [l7|. □ 

The above result reveals a special property of the uni- 
form ensemble: no matter what incomplete measurement 
is performed on it, the information which is not retrieved 
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by the measurement can always be extracted by subse- 
quent measurements. To see this we first use the polar 
decomposition theorem as before to write Aj = Uj^fWj. 

The final state is then given by p^ = UjEjJjj /Tr[Ej}. It 
is convenient to write the final ensemble as a distribu- 
tion over unormalized states, \<f>j). Writing these states 
in terms of the final state p'j, we have 

\h) = \[f>jWi)> (is) 

where = Uj\ip). The probability density of these 
states in the final ensemble (with respect to the Haar 
measure) is P(\<pj)) = (4>j\(f>j). Since the ensemble of 
states \ip) is uniform, so is the ensemble of states 

The final ensemble, Sj = {P(\<f>j)),\<pj}}, is referred to 
as a 'distortion' of the uniform ensemble by the state 
p'j . Since JRW have shown that such a distortion has an 
accessible information equal to Q[p'j], all the information 
missing in the incomplete measurement is accessible in 
the final ensembles Ej. 

While the dual Holevo bound is saturated for pure- 
state ensembles which maximize the accessible informa- 
tion (and measurements whose operators commute with 
the ensemble state) , the bound given by Eq. I|15|l is satu- 
rated by pure-state ensembles which minimize the acces- 
sible information. 

Since we have considered so far only efficient quantum 
measurements, we complete our discussion by examining 
classical measurements and inefficient quantum measure- 
ments. For this purpose it is best that we first introduce 
the latter. Inefficient measurements are simply efficient 
measurements in which the observer knows only that one 



of a subset of the possible results was obtained. As a re- 
sult, the observers final state of knowledge is given by 
averaging over a subset of the states p' y Thus, if we 
now label the measurement results by two indices, k and 
I, then in general we can write the actual final states 
for an observer who makes an inefficient measurement as 
Pk = YjiAkipA\jQk, where Qk is the probability that 
the final state is pk- 

Now, classical measurements are described by the sub- 
set of quantum measurements in which all the encoding 
states, pi, and all the measurement operators, Aj, are 
mutually commuting (for a discussion, see e.g. 12] ). Asa 
result it is easily shown that inefficient classical measure- 
ments are merely efficient classical measurements, and 
thus Eq.(|SJ remains true for all classical measurements. 
In fact, if the encoding states are pure classical states (i.e. 
individual classical states rather than distributions), then 
the bound is always saturated with equality. 

For inefficient quantum measurements however, Eq.ljSJ) 
does not hold. The reason for this is that for inefficient 
measurements Aif can be negative (whereas A/j is always 
non-negative) . An example of such a situation is one in 
which the initial state p is not maximally mixed, and 
the observer performs a von Neumann measurement in 
a basis unbiased with respect to the eigenbasis of p. If 
the observer has no knowledge of the outcome, then her 
final state is maximally mixed. Further, if one mixes, 
(in the sense of |19|) this measurement with one whose 
measurement operators commute with p, it is not hard 
to obtain a measurement in which both Ali and Aif are 
positive, but which violates Eq.©. 

The author is grateful to Michael J. W. Hall for helpful 
discussions. 
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